1. Field of the Invention
Embodiments described herein relate to color balancing techniques, and more specifically to stabilizing the illumination and color between frames of a video based on comparisons between the frames and a reference image.
2. Description of the Related Art
Computer graphics technology has come a long way since video games were first developed. Relatively inexpensive 3D graphics engines now provide nearly photo-realistic interactive game play on hand-held video game, home video game and personal computer hardware platforms costing only a few hundred dollars. These video game systems typically include a hand-held controller, game controller, or, in the case of a hand-held video game platform, an integrated controller. A user or player uses the controller to send commands or other instructions to the video game system to control a video game or other simulation being played. For example, the controller may be provided with a manipulator (e.g., a joystick) and buttons operated by the user.
Digital images are used in a wide array of different applications. In many applications, some form of image analysis and manipulation can be performed to enhance the picture quality of the digital images. For example, the quality of a digital image may be dependent on color levels and color intensity within the image, as unrealistic coloration within an image may degrade the visual quality of the image. When capturing a digital photo (e.g., with a digital camera device or a mobile device configured with a digital camera), characteristics of the ambient environment, e.g., lighting, reflections, etc., may impact the coloration of the digital picture, and in some cases may result in an unrealistic coloration for certain objects in the digital picture.
To address this, the coloration of a photographically recorded digital image can be adjusted using various color balancing techniques. Generally, color balancing involves altering the coloration of at least part of an image (or a frame in a video) in a deliberate fashion. However, while many imperfections may be corrected through the use of color balancing techniques, which aspects of the image, e.g., hue, saturation, chroma, colorfulness, value or lightness, luminance, etc., need to be adjusted to create realistic coloration can vary dramatically.
Unfortunately, many conventional color balancing techniques may provide inadequate results. For instance, because the coloration adjustments that need to be made vary so much from one image to another, many color balancing techniques rely on some form of manual user input or intervention in order to achieve a suitable color balance. As another example, other color balancing techniques may apply a predetermined color balancing operation to an image, and may then display the altered image to a user. In such a system, the user can then manually change the coloration of the image as is necessary to produce a more realistic image (e.g., an image whose coloration more realistically reflects the coloration of the original scene captured by the image). While these techniques may produce images with realistic coloration, they are ill-suited for realtime applications, such as color balancing frames within a video stream being displayed in realtime. In addition to the real-time aspect, these manual interfaces are generally less intuitive to use. That is, with conventional techniques, abstract tools such as changing saturation, hue, contrast, gamma and more general “tone curves” are used for color balancing purposes. However, none of these tools allow a user to map colors within one image onto the colors of another image (e.g., a particular red coloration in image A should look like a slightly different red coloration in image B, a green coloration in image A should actually look like a blue coloration in image B, etc.).
When working with captured high speed video, one common challenge is the removal of flicker with respect to illumination and color differences between frames of the captured video. While not restricted only to high speed video, this issue is more pronounced in high speed video capture because of interference between the camera's capture frequency and the frequencies of the light sources in the physical environment. For instance, for a camera with a high enough frame rate, the camera sensor can capture changes in brightness of some lamps due to alternating current electricity. These changes in brightness can create an undesirable flickering effect when the captured video is subsequently viewed.